Execute examples in parallel - #900
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the documentation example-build pipeline to execute example conversions concurrently, aiming to reduce overall docs build time while keeping the generated toctree structure intact.
Changes:
- Refactors per-example conversion into a dedicated
_convert_example()helper. - Collects all example conversion tasks first, then executes them in parallel via
ThreadPoolExecutor. - Adds a
max_workersparameter to control conversion concurrency.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
e23fcc5 to
0268bec
Compare
0268bec to
dfdf30d
Compare
dfdf30d to
58fa424
Compare
74f7bdc to
8791128
Compare
8791128 to
0e3f154
Compare
0e3f154 to
a6a41bc
Compare
a6a41bc to
6b5b791
Compare
0ed859a to
68ea474
Compare
There was a problem hiding this comment.
are you sure the link in the PR description is right? It points to emdgroup latest, which is main
since this PR is not on main I dont expect to see its cosnequences reflected in there or what am I missing?
There was a problem hiding this comment.
I will make sure to double-check and post a corrected link.
There was a problem hiding this comment.
I have adjusted the link in the description.Please feel free to resolve here after you have taken another look
There was a problem hiding this comment.
I assume you validates this performs correctly on fork?
There was a problem hiding this comment.
I did, but I will double-check just to be sure.
This PR makes the doc building execute the examples in parallel, significantly speeding up the overall execution.
This has been achieved as follows:
_convert_example)Important: While this always improves the running time, it is hard to estimate by how much since this seems to depend on how much work load is currently done in other CI tests. I've seen the full doc building job to complete between 8 and 14 minutes, which is a speed-up in any case.
Compiled docs: See https://avhopp.github.io/baybe_dev/latest/. PLEASE TAKE A LOOK HERE at the examples - I looked at some of them and everything looked as expected, but please double-check that your favourite examples look like you expect them to look.
Stack created with GitHub Stacks CLI • Give Feedback 💬